Robot, control method of robot, and control device of robot

ABSTRACT

A robot includes a grasping unit and performs an action based on: first imaging information of the grasping unit which does not grasp an object to be grasped in a first point; second imaging information of the grasping unit which does not grasp the object to be grasped in a second point which is different from the first point; and third imaging information of the object to be grasped which is grasped by the grasping unit in the first point.

BACKGROUND

1. Technical Field

The present invention relates to a robot, a control method of a robot,and a control device of a robot.

2. Related Art

If a position and a posture of a grasped object deviate from an expectedposition and posture, when a robot performs an assembly operation, theassembly operation may be affected negatively, in many cases. In a unitwhich detects the deviation, it is effective to image the grasped objectby an imaging unit and calculate a relationship in position and posturesbetween a grasping unit and the object to be grasped, however, thedeviations in a coordinate system of the robot and a coordinate systemof the imaging unit as a reference are different from each otherdepending on a place, and accordingly, the deviated amounts are notuniform, and the expected accuracy is difficult to obtain with one kindof correction amount. Therefore, the position and the posture may beslightly deviated even after grasping of the object to be grasped, andthe grasped object may not be assembled.

For example, there is disclosed a tool position correction method of anarticulated robot of storing a position deviation amount in the databaseand using the value when performing the action (for example, seeJP-A-2006-82171).

However, in the tool position correction method disclosed inJP-A-2006-82171, the grasping unit is visible from the imaging unit whenmeasuring the position deviation amount, but this state may not berealized in a case of the arrangement of an imaging unit of a double armrobot, or the like.

SUMMARY

An advantage of some aspects of the invention is to solve at least apart of the problems described above, and the invention can beimplemented as the following forms or application examples.

Application Example 1

This application example is directed to a robot including a graspingunit, in which the robot performs an action based on: first imaginginformation of the grasping unit which does not grasp an object to begrasped in a first point; second imaging information of the graspingunit which does not grasp the object to be grasped in a second pointwhich is different from the first point; and third imaging informationof the object to be grasped which is grasped by the grasping unit in thefirst point.

According to this application example, since the relationship betweenthe grasping unit and the object to be grasped by the grasping unit iscalculated based on each imaging information item at the positions ofthe first point (for example, check position) and the second point (forexample, target position), it is possible to decrease the deviation (ofthe target position) at the time of assembly. Therefore, it is possibleto provide a robot which realizes assembly with high accuracy.

Application Example 2

In the robot according to the application example described above, it ispreferable that the first imaging information, the second imaginginformation, and the third imaging information are position and postureinformation items.

According to this application example, since each imaging information isthe position and the posture information, it is possible to easily checkthe position and the posture of the grasping unit and the object to begrasped.

Application Example 3

In the robot according to the application example described above, it ispreferable that the robot further includes a fourth control unit whichcompares the first imaging information, the second imaging information,and the third imaging information with each expected value which ispreviously set, and calculates each correction amount for correcting adeviation amount according to the results of the comparison.

According to this application example, since each imaging informationitem is corrected at the positions of the first point and the secondpoint, it is possible to decrease the deviation at the time of assembly.

Application Example 4

In the robot according to the application example described above, it ispreferable that the robot further includes an imaging unit, the graspingunit includes a plurality of claw portions each including a distalportion which is disposed in a first direction and a second directionand grasps the object to be grasped, and a proximal portion which ispositioned in a third direction orthogonal to the first direction andthe second direction with respect to the distal portion, and the postureof the grasping unit to be imaged by the imaging unit in the first pointis that the imaging unit is positioned in the third direction on thedistal portion side of the claw portion.

According to this application example, it is possible to expose a rearsurface (surface not overlapping the grasping unit and the claw portion)of the object to be grasped in the imaging direction of the imagingunit. Accordingly, there is no shielding between the imaging unit andthe object to be grasped, and the object to be grasped is rarelyscreened by the grasping unit. As a result, it is possible to image theoutline shape of the rear side of the object to be grasped.

Application Example 5

In the robot according to the application example described above, it ispreferable that a plurality of the imaging units are provided.

According to this application example, since the plurality of imagingunits are provided, it is possible to increase resolution of the imageobtained by the imaging unit and it is possible to create an imagehaving excellent accuracy. Accordingly, it is possible to obtain theposition and the posture having high positional accuracy.

Application Example 6

In the robot according to the application example described above, it ispreferable that the imaging unit is a stereo camera.

According to this application example, there is no shielding between theimaging unit and the object to be grasped, and the object to be graspedis rarely screened by the grasping unit.

Application Example 7

In the robot according to the application example described above, it ispreferable that the second imaging information uses the shape of thegrasping unit or an image of a marker provided on the grasping unit.

According to this application example, it is possible to easily performthe calculation of the second imaging information.

Application Example 8

This application example is directed to a control method of a robotwhich includes a grasping unit, including controlling the robot basedon: first imaging information of the grasping unit which does not graspan object to be grasped in a first point; second imaging information ofthe grasping unit which does not grasp the object to be grasped in asecond point which is different from the first point; and third imaginginformation of the object to be grasped which is grasped by the graspingunit in the first point.

According to this application example, since the relationship betweenthe grasping unit and the object to be grasped which is grasped by thegrasping unit is calculated based on each imaging information item inthe positions of the first point and the second point, it is possible todecrease the deviation at the time of assembly. Accordingly, it ispossible to provide a control method of a robot which controls a robotwhich realizes assembly with a high accuracy.

Application Example 9

In the control method of a robot according to the application exampledescribed above, it is preferable that the control method furtherincludes comparing the first imaging information, the second imaginginformation, and the third imaging information with each expected valuewhich is previously set, and calculating each correction amount forcorrecting a deviation amount according to the results of thecomparison.

According to this application example, since each imaging informationitem is corrected in the positions of the first point and the secondpoint, it is possible to decrease the deviation at the time of assembly.

Application Example 10

This application example is directed to a control device of a robotwhich controls a robot including a grasping unit based on: first imaginginformation of the grasping unit which does not grasp an object to begrasped in a first point; second imaging information of the graspingunit which does not grasp the object to be grasped in a second pointwhich is different from the first point; and third imaging informationof the object to be grasped which is grasped by the grasping unit in thefirst point.

According to this application example, since the relationship betweenthe grasping unit and the object to be grasped which is grasped by thegrasping unit is calculated based on each imaging information item inthe positions of the first point and the second point, it is possible todecrease the deviation at the time of assembly. Accordingly, it ispossible to provide a control method of a robot which controls a robotwhich realizes assembly with a high accuracy.

Application Example 11

In the control device of a robot according to the application exampledescribed above, it is preferable that the control device furtherincludes a fourth control unit which compares the first imaginginformation, the second imaging information, and the third imaginginformation with each expected value which is previously set, andcalculates each correction amount for correcting a deviation amountaccording to the results of the comparison.

According to this application example, since each imaging informationitem is corrected in the positions of the first point and the secondpoint, it is possible to decrease the deviation at the time of assembly.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIG. 1 is a diagram showing an example of a usage situation of a robotaccording to a first embodiment.

FIG. 2 is a diagram showing an example of a hardware configuration of acontrol device according to the first embodiment.

FIG. 3 is a diagram showing an example of a functional configuration ofthe control device according to the first embodiment.

FIG. 4 is a diagram illustrating an example of a grasping method of anobject with a hand according to the first embodiment.

FIG. 5 is a diagram showing a hand at a check position according to thefirst embodiment.

FIG. 6 is a diagram showing a hand at a target position according to thefirst embodiment.

FIG. 7 is a diagram showing a hand at a check position according to thefirst embodiment.

FIG. 8 is a flowchart showing an example of a flow of a preprocessexecuted by a robot control unit of the robot according to the firstembodiment.

FIG. 9 is a flowchart showing an example of a flow of an assemblyprocess executed by the robot control unit of the robot according to thefirst embodiment.

FIGS. 10A and 10B are diagrams showing position and postures of a handaccording to the first embodiment, in which FIG. 10A is a diagramshowing a target position of a hand and FIG. 10B is a diagram showingcorrection of a position and a posture of a hand.

FIG. 11 is a diagram showing a movement position of a hand according toa second embodiment.

FIG. 12 is a flowchart showing an example of a flow of an assemblyprocess executed by a robot control unit of a robot according to thesecond embodiment.

DESCRIPTION OF EXEMPLARY EMBODIMENTS First Embodiment

Hereinafter, specific embodiments of the invention will be describedwith reference to the drawings. The drawings used herein show suitablyenlarged or contracted illustrated parts, so that the illustrated partscan be recognized.

FIG. 1 is a diagram showing an example of a usage situation of a robotaccording to the embodiment.

A robot 2 according to the embodiment includes a camera (imaging unit)10 and a control device 12. The imaging unit 10 is mounted on the robot2. The camera 10 images a grasped object OBJ (object to be grasped) (seeFIG. 7). The camera 10 performs the imaging as a position and a postureof the object OBJ grasped by a hand (grasping unit) HND1 are changed.

A plurality of cameras 10 may be provided. According to this, byproviding the plurality of cameras 10, it is possible to increaseresolution of an image acquired by the cameras 10 and to create an imagehaving an excellent accuracy. Accordingly, it is possible to obtain aposition and a posture (three-dimensional position and posture) having ahigh positional accuracy.

The camera 10 is, for example, a camera including a charge-coupleddevice (CCD) or a complementary metal oxide semiconductor (CMOS) whichis an imaging device converting condensed light into an electric signal.The camera 10 is, for example, a stereo camera configured with twocameras, but may be configured with three or more cameras, or may imagea two-dimensional image with one camera. The camera is a movable typeand is movable vertically and horizontally. According to this, there isno shielding between the camera 10 and the object OBJ and the object OBJis rarely screened by the hand HND1.

The camera 10 is, for example, connected to the control device 12through a cable to communicate with the control device. The wiredcommunication through the cable is performed based on standards such asEthernet (registered trademark) or universal serial bus (USB), forexample. The camera 10 and the control device 12 may be connected toeach other through wireless communication performed based oncommunication standards such as Wi-Fi (registered trademark). The objectOBJ is previously installed on an installation surface M. The“installation surface M” is, for example, a surface on a table. Thecamera 10 is installed at a position so as to image the object OBJ. Thecamera 10 images the object OBJ and the captured image of the imagedobject OBJ is output to the communication device 12 throughcommunication.

As shown in FIG. 1, the robot 2 is, for example, a double arm robotincluding the hand HND1, a hand HND2, force sensors 14, an arm portionARM1, an arm portion ARM2, and a plurality of actuators (not shown), onright and left arms which are base shafts of the robot. Each of the baseshafts of the robot includes a rotation shaft and rotates. Each arm ofthe robot 2 is a six-axial vertical articulated type. A support table,the arm portion ARM1, and the grasping unit HND1 of one arm can performan action having degrees of freedom in six axes by a joint action by theactuators, and a support table, the arm portion ARM2, and the hand HND2of the other arm can perform an action having a degree of freedom withsix axes by a joint action by the actuators. Each arm of the robot 2 mayperform an action with five or less degrees of freedom (five axes) ormay perform an action with seven or more degrees of freedom (sevenaxes). Hereinafter, the action of the robot 2 performed by the armincluding the hand HND1 and the arm portion ARM1 will be described, butthe same action may be performed by the arm including the hand HND2 andthe arm portion ARM2. The hand HND1 grasps the object OBJ. The “handHND1” is an example of the “grasping unit” in the appended claims. Therobot 2 is, for example, connected to the control device 12 through acable to communicate with the control device. The wired communicationthrough the cable is performed based on standards such as Ethernet(registered trademark) or USB, for example. The robot 2 and the controldevice 12 may be connected to each other through wireless communicationperformed based on communication standards such as Wi-Fi (registeredtrademark). The robot 2 shown in FIG. 1 is a double arm robot, but maybe realized as a single arm robot.

The hand HND1 of the robot 2 includes claw portions 52 which can graspor clamp the object OBJ (see FIG. 4). A force sensor 14 is providedbetween the hand HND1 and the arm portion ARM1 of the robot 2 anddetects a force or a moment operating on the hand HND1. The force sensor14 outputs information regarding the detected force or moment to thecontrol device 12 through the communication. The information regardingthe force or moment detected by the force sensor 14 is, for example,used in impedance control of the robot 2 by the robot control unit 16.

The robot 2 acquires a control signal based on the three-dimensionalposition and posture of the object OBJ from the control device 12 andperforms a predetermined operation for the object OBJ based on theacquired control signal. The predetermined operation is, for example, anoperation of grasping the object BOJ by the hand HND1 of the robot 2 andmoving the grasped object OBJ from the current position to anotherposition or assembling the object into another device after themovement.

The camera 10 may be installed at a portion other than the hand HND1which grasps the object OBJ. According to this, it is possible to avoida situation where only a partial shape of the object OBJ can be imageddue to the positional relationship between the object OBJ and the camera10 approaching each other, when the object OBJ is grasped.

The control device 12 performs image processing of the image captured bythe camera 10. The control device 12 calculates the position and theposture of the object OBJ. The control device 12 performs control suchthat the robot 2 performs a predetermined operation. More specifically,the control device 12 calculates the three-dimensional position and theposture of the object OBJ based on the captured image of the object OBJimaged by the camera 10. The control device 12 causes the robot 2 tograsp the object OBJ by the hand HND1, based on the calculatedthree-dimensional position and the posture of the object OBJ. Afterthat, the control device 12 controls the robot 2 so that the robotperforms a predetermined operation with respect to the grasped objectOBJ.

Next, the hardware configuration of the control device 12 will bedescribed with reference to FIG. 2.

FIG. 2 is a diagram showing an example of the hardware configuration ofthe control device 12 according to the embodiment. The control device12, for example, includes a central processing unit (CPU) 20, a storageunit 22, an input reception unit 24, and a communication unit 26, andcommunicates with the camera 10 or the like through the communicationunit 26. The constituent elements are connected to each other through abus so as to communicate with each other. The CPU 20 executes variousprograms stored in the storage unit 22. The storage unit 22, forexample, includes a hard disk drive (HDD) or solid state drive (SSD), anelectrically erasable programmable read-only memory (EEPROM), aread-only memory (ROM), a random access memory (RAM), or the like, andstores various information items or images and programs to be processedby the control device 12. The storage unit 22 may be an external storagedevice connected by a digital input and output port of the USB or thelike, instead of the unit built into the control device 12.

The input reception unit 24 is, for example, a keyboard or a mouse, atouch pad, and other input devices. The input reception unit 24 mayfunction as a display unit and may be configured as a touch panel. Thecommunication unit 26 is, for example, configured to include a digitalinput and output port of the USB or Ethernet (registered trademark)port.

Next, the functional configuration of the control device 12 will bedescribed with reference to FIG. 3.

FIG. 3 is a diagram showing an example of the functional configurationof the control device 12 according to the embodiment. The control device12, for example, includes an image acquisition unit 30, athree-dimensional position and posture derivation unit 32, and the robotcontrol unit 16. Some or all of the functional units are realized byexecuting various programs stored in the storage unit 22 by the CPU 20,for example. Some or all of the functional units may be hardwarefunctional units such as the large scale integration (LSI) or anapplication specific integrated circuit (ASIC).

The image acquisition unit 30 acquires the image captured by the camera10 and outputs the acquired captured image to the three-dimensionalposition and posture derivation unit 32. The image acquisition unit 30stores the acquired captured image in the storage unit 22, and thethree-dimensional position and posture derivation unit 32 may read thecaptured image from the storage unit 22. The three-dimensional positionand posture derivation unit 32 derives the three-dimensional positionand posture of the object OBJ based on the captured image acquired fromthe image acquisition unit 30. The three-dimensional position andposture derivation unit 32 outputs the derived three-dimensionalposition and the posture of the object OBJ to the robot control unit 16.

The robot control unit 16 controls the robot 2 so as to cause the robot2 to grasp the object OBJ by the hand HND1, based on thethree-dimensional position and the posture of the object OBJ acquiredfrom the three-dimensional position and posture derivation unit 32.

The robot control unit 16 includes a registration section, a graspingsection, a movement section, a correction section, and an assemblysection.

The registration section registers a target position of the hand HND1 sothat the object OBJ has the position and the posture to be assembled onan assembly surface MS (see FIG. 10A).

The grasping section grasps the object OBJ by the hand HND1.

The movement section causes an outline shape of the object OBJ to moveto a position so as to be imaged by the camera 10, in a state where theobject OBJ is grasped (see FIG. 11).

The correction section corrects the target position of the hand HND1 sothat the object OBJ has the position and the posture to be assembled onan assembly surface MS (see FIG. 10B).

The assembly section assembles the object OBJ to the assembly surface MS(see FIG. 10B).

The robot control unit 16 includes a first control unit, a secondcontrol unit, a third control unit, and a fourth control unit.

The first control unit calculates position and posture information(first imaging information) using an image of the hand HND1 not graspingthe object OBJ which is imaged by the camera 10 at a check position(first point) set on a movement path of the hand HND1.

The second control unit calculates position and posture information(second imaging information) using an image of the hand HND1 notgrasping the object OBJ which is imaged by the camera 10 at a targetposition (second point) set at a position different from the checkposition on the movement path.

The third control unit calculates position and posture information(third imaging information) using an image of the object OBJ grasped bythe hand HND1 which is imaged by the camera 10 at the check position.

The fourth control unit compares each position and posture informationitem with each previously set expected value and calculates eachcorrection amount for correcting the deviation amount according to theresults of the comparison. According to this, since each position andposture information item is corrected regarding the check position andthe target position, it is possible to decrease deviation at the time ofassembly. The database of the correction amounts may be acquired fromthe movement path (operation range) of the hand HND1. According to this,the error is decreased, and it is possible to perform the assembly withthe grasping error, even after grasping the object OBJ.

FIG. 4 is a diagram illustrating an example of a grasping method of theobject OBJ with the hand HND1 according to the embodiment. The upperpart of the diagram of FIG. 4 is a cross-sectional view of a positionalrelationship between the hand HND1 and the object OBJ installed on theinstallation surface M, when the installation M is viewed from the rightside. The lower part of the diagram of FIG. 4 is a diagram of apositional relationship between the hand HND1 and the object OBJinstalled on the installation surface M, when the installation M is seenfrom the top (for example, side of the surface where the object OBJ isinstalled). As long as the installation surface M is the surface wherethe object OBJ is installed, the installation surface is not necessarilya surface orthogonal to a vertical direction like a table surface, andmay be a wall surface, for example.

As shown in FIG. 4, the hand HND1 includes the plurality of clawportions 52 each including a distal portion 54 which is disposed in afirst direction (x) and a second direction (y) and grasps the objectOBJ, and a proximal portion 56 which is positioned in a third direction(z) orthogonal to the first direction (x) and the second direction (y)with respect to the distal portion 54.

Herein, the grasping method of the object OBJ by the hand HND1 will bedescribed with reference to FIG. 4.

First, the robot control unit 16 moves the hand HND1 of the robot 2 tothe position on the installation surface M shown in the lower sidedrawing of FIG. 4, based on the three-dimensional position and postureof the object OBJ. Herein, a coordinate axis of the upper and lower sidedrawings of FIG. 4 is a coordinate axis for showing the positionalrelationship between the installation surface M and the object OBJ andis not a robot coordinate system or a coordinate axis on the capturedimage. In a case where the object OBJ is installed on the installationsurface M, the position of the hand HND1 shown in the lower side drawingof FIG. 4 is the top of the object OBJ. The robot control unit 16 movesthe object OBJ to the place so as to be grasped by the hand HND1, bymoving the hand HND1 in the direction (z direction) shown with an arrowin the upper side drawing of FIG. 4.

The robot control unit 16 controls the robot 2 so that the robotperforms a predetermined operation, after the hand HND1 grasps theobject OBJ.

Preprocess

FIG. 5 is a diagram showing the hand HND1 at the check positionaccording to the embodiment. FIG. 6 is a diagram showing the hand HND1at the target position according to the embodiment.

First, the hand HND1 is moved to the check position for checking theobject OBJ to be grasped in a state where the object OBJ is not grasped,and the position and the posture information (first imaging information)of the hand HND1 is acquired. Accordingly, it is possible to acquire theposition and the posture information of the hand HND1 at the positionfor checking the object OBJ to be grasped. The position and the postureinformation is, for example, three-dimensional position and postureinformation. As shown in FIG. 5, in the checking of the position and theposture information of the hand HND1, a position and a posture of a palmof the hand HND1 is acquired. In the checking of the position and theposture information of the hand HND1, a portion shown with an arrow A ofFIG. 5 may be detected. The position to which the hand HND1 is moved,may be on a linear line obtained by connecting an intersection ofoptical axes of the cameras 10 and a base line between the cameras 10.It is possible to understand a relationship between the position and theposture information of the hand HND1 acquired with direct kinematics andposition and posture information of the hand HND1 in the cameracoordinate system.

Next, the hand HND1 is moved to the target position around the assemblysurface MS and the position and the posture information (second imaginginformation) of the hand HND1 is acquired. In the checking of theposition and the posture information of the hand HND1, a part shown withan arrow B of FIG. 6 may be measured. At a position for performing theassembly, it is difficult to observe the hand HND1 from the stereocamera on the head portion, due to shielding of the arm, in some cases.Accordingly, as shown in FIG. 6, a jig is used so that a part of a jig34 can be observed from the camera 10. For example, a marker 36 isattached to a part of the jig 34, and the marker 36 is measured by thecamera 10. In this case, a linear line from the two markers 36 of thejig 34 is calculated. The jig 34 is held in two directions and anintersection of two linear lines is acquired, and accordingly, this isset as the position of the hand HND1. Since the surface is configuredwith two linear lines, the direction of this surface can be set as adirection of the palm of the hand HND1. In the same manner as describedabove, it is possible to understand a relationship between the positionand the posture information of the hand HND1 acquired with directkinematics and position and posture information of the hand HND1 in thecamera coordinate system. According to this, it is possible to easilycalculate the position and the posture information of the hand HND1.

At the Time of Actual Action

FIG. 7 is a diagram showing the hand HND1 at the check positionaccording to the embodiment.

In a state where the object OBJ is grasped, the hand HND1 is moved tothe check position for checking the object OBJ, and the position and theposture information (third imaging information) of the grasped objectOBJ is acquired. Accordingly, it is possible to acquire a relationshipbetween the position and the posture information of the hand HND1 andthe position and the posture information of the grasped object OBJ, inthe camera coordinate system. As shown in FIG. 7, in the checking of theposition and the posture information of the hand HND1, a position and aposture of a palm of the hand HND1 is acquired.

The correction of the position and the posture information is performedusing the relationship with the position and the posture information ofthe hand HND1 acquired with direct kinematics which is acquired inadvance preparation. It is assumed that there is a repeat accuracy ofthe robot.

Next, a tip of the hand HND1 is moved to the target position. Themovement amount is set by considering the correction amount. The movedtip has the position and the posture obtained by considering thecorrection amount calculated in the advance preparation.

Regarding the posture of the hand HND1 imaged by the camera 10 at thecheck position, the camera 10 is preferably positioned of the clawportion 52 in the third direction (z) on the distal portion 54 side.According to this, it is possible to display a rear surface (surface notoverlapping the hand HND1 or the claw portion 52) of the object OBJ inthe imaging direction of the camera 10. Accordingly, there is noshielding between the camera 10 and the object OBJ and the object OBJ israrely screened by the hand HND1. As a result, it is possible to imagethe outline shape of the rear side of the object OBJ.

EXAMPLE

As an example, an operation of assembling a screw fastening plate to ascrew fastening base is used.

FIG. 8 is a flowchart showing an example of a flow of a preprocessexecuted by the robot control unit of the robot according to theembodiment. FIG. 9 is a flowchart showing an example of a flow of anassembly process executed by the robot control unit of the robotaccording to the embodiment. FIGS. 10A and 10B are diagrams showing theposition and the postures of the hand HND1 according to the embodiment.FIG. 10A is a diagram showing the target position of the hand HND1 andFIG. 10B is a diagram showing correction of the position and the postureof the hand HND1. The object OBJ is a screw fastening plate and theassembly surface MS is an assembly surface of a screw fastening base. Wis a world coordinate and T denotes coordinate conversion.

Hereinafter, a process performed by the robot control unit 16 when therobot 2 assembles the object OBJ to the assembly surface MS will bedescribed with reference to FIG. 8. Hereinafter, the hand HND1 of therobot 2 will be described with the assumption that the hand is moved tothe immediately above the object OBJ by the robot control unit 16.

First, as shown in FIG. 8, in Step S10, the hand HND1 is moved to thecheck position for checking the object OBJ to be grasped.

In Step S20, the position and the posture information (first imaginginformation) of the check position (first point) of the hand HND1 isacquired. FIG. 10B shows this operation as ^(W)T_(HND1′). According tothis, since the first imaging information is the position and theposture information, it is possible to easily check the position and theposture of the hand HND1.

In Step S30, the position and the posture information of the checkposition of the hand HND1 is registered as a previous position andposture 1.

In Step S40, the hand HND1 is moved to the vicinity of the targetposition for assembling the grasped object OBJ.

In Step S50, the position and the posture information (second imaginginformation) of the target position (second point) of the hand HND1 isacquired. FIG. 10B shows this operation as ^(W)T_(HND1). According tothis, since the second imaging information is the position and theposture information, it is possible to easily check the position and theposture of the hand HND1.

In Step S60, position and posture information of the target position ofthe hand HND1 is registered as a previous position and posture 2.

Hereinafter, the process performed by the robot control unit 16 when therobot 2 assembles the object OBJ to the assembly surface MS will bedescribed with reference to FIG. 9. Hereinafter, the hand HND1 of therobot 2 will be described with the assumption that the hand is moved tothe immediately above the object OBJ by the robot control unit 16.

First, as shown in FIG. 9, in Step S110, the object OBJ is registered.Specifically, the position and the posture of the object OBJ withrespect to the assembly surface MS is designated. FIG. 10B shows thisoperation as ^(MS)T_(OBJ). The position and the posture of the assemblysurface MS in world coordinates W is designated. FIG. 10B shows thisoperation as ^(W)T_(MS). The position and the posture of the object OBJin world coordinates W is calculated. For example, this operation can beexpressed as ^(W)T_(OBJ)=^(W)T_(MS) ^(MS)T_(OBJ). The position and theposture information of the check position of the object OBJ is acquired.For example, FIG. 10B shows this operation as ^(W)T_(OBJ′).

In Step S120, the object OBJ is grasped.

In Step S130, the hand HND1 is moved to the check position for checkingthe grasped object OBJ.

In Step S140, the position and the posture information (third imaginginformation) of the grasped object OBJ is acquired. According to this,since the third imaging information is the position and the postureinformation, it is possible to easily check the position and the postureof the object OBJ.

In Step S150, the position and the posture of the hand HND1 is correctedusing the previous position and posture 1 of the hand HND1. Thecorrection amount of the check position of the hand HND1 is calculatedusing the previous position and posture 1 of the hand HND1. FIG. 10Bshows this operation as T_(HND1′). The check position of the hand HND1is corrected. For example, this operation can be expressed as T_(HND1′)^(W)T_(HND1′).

In Step S160, a relationship between the position and the posture of thehand HND1 and the position and the posture of the grasped object OBJ iscalculated.

In Step S170, the position of the hand HND1 from which the graspedobject OBJ is moved to the target position is calculated. The conversionamount for moving the object OBJ to the target position is calculated.For example, this operation can be expressed as^(W)T_(OBJ′)=T^(W)T_(OBJ) and T=^(W)T_(OBJ′) ^(W)T_(OBJ) ⁻¹. The targetposition of the hand HND1 is calculated. For example, this operation canbe expressed as ^(W)T_(HND1)=TT_(HND1′) ^(W)T_(HND1′).

In Step S180, the position and the posture of the hand HND1 is correctedusing the previous position and posture 2 of the hand HND1. Thecorrection amount of the target position of the hand HND1 is calculatedusing the previous position and posture 2 of the hand HND1. FIG. 10Bshows this operation as T_(HND1). The target position of the hand HND1is corrected. For example, this operation can be expressed as T_(HND1)^(W)T_(HND1).

In Step S190, the hand HND1 is moved to the corrected target position.

In Step S200, the assembly operation is performed. According to this, itis possible to move the screw fastening plate as close as possible tothe desired position.

As described above, since the robot 2 of the embodiment calculates therelationship between the hand HND1 and the object OBJ grasped by thehand HND1 based on each position and posture information item regardingthe check position and the target position, it is possible to decreasedeviation at the time of assembly. Therefore, it is possible to providethe robot 2, the control method of the robot 2, and the control device12 of the robot 2 which realize assembly with high accuracy.

Second Embodiment

The embodiment is different from that of the first embodiment in a pointthat the robot 2 of the embodiment performs the imaging by temporarilychanging the position and the posture of the grasped object OBJ to be inan imaging area of the camera 10. Hereinafter, the same referencenumerals are used for the same configuration members as those of thefirst embodiment, and therefore the description thereof will be omittedor simplified herein.

FIG. 11 is a diagram showing a movement position of the hand HND1according to the embodiment.

The plurality of cameras 10 may be configured with two cameras 10. Theobject OBJ grasped by the hand HND1 may be on a vertical line 44 drawnfrom an intersection 42 of optical axes 40 of the two cameras 10 to alinear line 48 connecting the positions of the two cameras 10.

In addition, the center of gravity 46 of the object OBJ grasped by thehand HND1 may be on the vertical line 44 drawn from the intersection 42of the optical axes 40 of the two cameras 10 to the linear line 48connecting the positions of the two cameras 10.

An opening and closing direction of the claw portion 52 of the hand HND1grasping the object OBJ may be a normal line direction of a plane 50including the intersection 42 of the optical axes 40 of the two cameras10 and the position of the two cameras 10. According to this, there isno shielding between the cameras 10 and the object OBJ, and the objectOBJ is rarely screened by the hand HND1.

Hereinafter, a process performed by the robot control unit 16 when therobot 2 assembles the object OBJ to the assembly surface MS will bedescribed with reference to FIG. 12.

FIG. 12 is a flowchart showing an example of a flow of the assemblyprocess executed by the robot control unit 16 of the robot 2 accordingto the embodiment.

First, as shown in FIG. 12, in step 210, the target position of the handHND1 is registered in the storage unit 22 as shown in FIG. 10A, so thatthe object OBJ has the position and the posture to be assembled to theassembly surface MS, as a registration step.

In Step S220, data for detecting the object OBJ and the installationsurface M is registered in the storage unit 22. Alternatively, thethree-dimensional position and posture of the object OBJ with respect tothe installation surface M is registered in the storage unit 22.

In Step S230, the three-dimensional position and posture of theinstallation surface M and the object OBJ is acquired from the storageunit 22.

In Step S240, the object OBJ is grasped using the hand HND1 as agrasping step. The position and the posture where the grasped object OBJcan be observed by the cameras 10 is calculated.

In Step S250, the object OBJ is moved to the position so as to be imagedby the camera 10, in a state where the object OBJ is grasped, as amovement step. The grasped object OBJ is moved to in front of thecameras 10 and the three-dimensional position and posture of the graspedobject OBJ is detected.

In Step S260, the three-dimensional position and posture of the objectOBJ is acquired by the cameras 10.

In Step S270, the position and the posture of the hand HND 1 graspingthe object OBJ is acquired from the storage unit 22.

In Step S280, a relationship (deviation from a set value) of thethree-dimensional position and posture between the object OBJ and thehand HND1 is acquired.

In Step S290, the three-dimensional position and posture of the assemblysurface MS is acquired from the storage unit 22.

In Step S300, the target position of the hand HND1 is corrected so thatthe object OBJ has the position and the posture to be assembled to theassembly surface MS, as a correction step. The target position of thehand HND1 with the corrected deviation from the set value is calculated.The position and the posture of the hand HND1 is changed so that thegrasped object OBJ is at the target position.

In Step S310, the object OBJ is assembled to the assembly surface MS asan assembly step. By performing the correction of the position and theposture of the hand HND1 regarding the inclination (deviation) of theobject OBJ with respect to the hand HND1, it is possible to have minimuminclination of the object OBJ with respect to the assembly surface MS,when performing the assembly, as shown in FIG. 10B. The process thenends.

As described above, the robot 2 according to the embodiment performs theimaging by temporarily changing the position and the posture of thegrasped object OBJ to be in the imaging area of the camera 10.Accordingly, it is possible to perform the imaging in a state where theamount of the object OBJ shielded by the hand HND1 of the robot 2 isdecreased as much as possible, at the time of the imaging. It ispossible to ascertain the positional relationship between the object OBJand the hand HND1 by converting the camera coordinates of the object OBJto the world coordinate. As a result, it is possible to perform theassembly with little errors by correcting the position deviation betweenthe object OBJ and the hand HND1 in a state where the robot 2 grasps theobject OBJ.

When the relationship in position and postures between the hand HND1 andthe camera 10 or the direction of the line of sight of the camera 10 isalready known, the object OBJ may be moved so as to be in the sight ofthe camera 10, in order to check the state of the object OBJ grasped bythe robot. At that time, it is desirable that there is as littleshielding as possible between the camera 10 and the object OBJ, and theobject OBJ is rarely screened by the hand HND1.

By comparing the relationship between the detected position and postureof the object OBJ and the position and the posture of the hand HND1, itis possible to detect whether or not this relationship has deviated fromthe expected relationship registered in the database.

Since the imaging unit is set to be movable by rotating the head andbody, it is possible to image the target without taking an unusualposture, and it is possible to accurately acquire the position and theposture of the target.

The embodiments are not limited to the above descriptions, and thefollowing embodiments can be used.

In the embodiments described above, the opening and closing direction ofthe claw portion 52 of the hand HND1 is not limited to coincide with thenormal line of the plane 50 including the intersection 42 of the opticalaxes 40 of the imaging unit 10 and the positions of the two cameras 10,and an angle formed by the two may be approximately 90 degrees dependingon the grasped state, for example.

Since the shape of the object OBJ to be detected is already known, therelationship (deviation from a set value) in the three-dimensionalposition and posture between the object OBJ and the hand HND1 may beacquired from the image of a part of the object OBJ.

Since the optical axes 40 rarely intersect with each other, theintersection 42 of the optical axes 40 of the cameras 10 may becalculated by a midpoint method or the like.

By performing the operation in combination with a visual servo, anoperation start position of the visual servo is stable, and accordingly,it is possible to expect assembly with a high accuracy.

Hereinabove, the robot 2, the control method of the robot 2, and thecontrol device 12 of the robot 2 have been described based on theembodiments shown in the drawings, but the invention is not limitedthereto, and the configuration of each unit can be replaced with anarbitrary configuration having the same function. In addition, otherarbitrary constituent elements may be added to the invention.

The entire disclosure of Japanese Patent Application No. 2014-113819,filed Jun. 2, 2014 and 2015-034917, filed Feb. 25, 2015 are expresslyincorporated by reference herein.

What is claimed is:
 1. A robot comprising a grasping unit, wherein therobot performs an action based on: first imaging information of thegrasping unit which does not grasp an object to be grasped in a firstpoint; second imaging information of the grasping unit which does notgrasp the object to be grasped in a second point which is different fromthe first point; and third imaging information of the object to begrasped which is grasped by the grasping unit in the first point.
 2. Therobot according to claim 1, wherein the first imaging information, thesecond imaging information, and the third imaging information areposition and posture information items.
 3. The robot according to claim1, further comprising: a fourth control unit which compares the firstimaging information, the second imaging information, and the thirdimaging information with each expected value which is previously set,and calculates each correction amount for correcting a deviation amountaccording to the results of the comparison.
 4. The robot according toclaim 1, further comprising: an imaging unit, wherein the grasping unitincludes a plurality of claw portions each including a distal portionwhich is disposed in a first direction and a second direction and graspsthe object to be grasped, and a proximal portion which is positioned ina third direction orthogonal to the first direction and the seconddirection with respect to the distal portion, and the posture of thegrasping unit to be imaged by the imaging unit in the first point isthat the imaging unit is positioned in the third direction on the distalportion side of the claw portion.
 5. The robot according to claim 4,wherein a plurality of the imaging units are provided.
 6. The robotaccording to claim 4, wherein the imaging unit is a stereo camera. 7.The robot according to claim 1, wherein the second imaging informationuses the shape of the grasping unit or an image of a marker provided onthe grasping unit.
 8. A control method of a robot which includes agrasping unit, comprising: controlling the robot based on: first imaginginformation of the grasping unit which does not grasp an object to begrasped in a first point; second imaging information of the graspingunit which does not grasp the object to be grasped in a second pointwhich is different from the first point; and third imaging informationof the object to be grasped which is grasped by the grasping unit in thefirst point.
 9. The control method of a robot according to claim 8,further comprising: comparing the first imaging information, the secondimaging information, and the third imaging information with eachexpected value which is previously set, and calculating each correctionamount for correcting a deviation amount according to the results of thecomparison.
 10. A control device of a robot which controls a robotincluding a grasping unit based on: first imaging information of thegrasping unit which does not grasp an object to be grasped in a firstpoint; second imaging information of the grasping unit which does notgrasp the object to be grasped in a second point which is different fromthe first point; and third imaging information of the object to begrasped which is grasped by the grasping unit in the first point. 11.The control device of a robot according to claim 10, further comprising:a fourth control unit which compares the first imaging information, thesecond imaging information, and the third imaging information with eachexpected value which is previously set, and calculates each correctionamount for correcting a deviation amount according to the results of thecomparison.